As the World Wide Web (WWW or Web) increases in popularity, traffic over the Internet has increased considerably. The Web has now become one of the primary bottlenecks on network performance. For example, a Web user may experience noticeable delays when documents or information are requested by the user who is connected to a server via a slow network link. The Web's transferring of the information over the network to the user further increases the level of traffic over the network. The increased traffic reduces the bandwidth available for the requests of other users, contributing to additional potential delays.
To reduce access latencies for clients, the prior art has stored or cached copies of popular documents of information closer to the user at network nodes, from which the access latencies are more acceptable. The caching can be implemented at various points on the network. For example, a large university or corporation may have its own local cache, from which all the users subscribing to that network may fetch documents. A local cache may be implemented in the form of a specialized server.
A specialized server, which is called a caching proxy, may act as an agent on the behalf of clients to locate any potential cached copy of requested information. Caching proxies usually serve as secondary or higher level caches because caching proxies are concerned only with misses left over from failed client caches. Client caches are built into various Web browsers. Client caches may either store only the document accesses during the current invocation (nonpersistent cache such as Mosaic) or may cache documents across invocations.
Previous work on the caching of Web documents has mainly been in the area of on-demand caching. On-demand caching means a caching decision is made after an object is requested. For example, Abrams et. al., "Caching Proxies: Limitations and Potentials", Proc. 4th International World Wide Web Conference, 1996, describes different on-demand caching policies that have been examined for the Web.
In an Internet Service Provider (ISP) network or a corporate network, different regional proxies or departmental proxies in the proxy hierarchy may observe different reference behavior or face different network traffic and delay. The presence of the proxy hierarchy further complicates the prefetch decision on whether all the intermediate proxies, the client, or a subset of them should perform the prefetch. Thus, a need exists for a prefetching method and system that can effectively coordinate prefetching activities with respect to the proxy server hierarchy and associated clients.